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ABSTRACT 

This study investigated the return on investment (ROI) of the 
Southern extension Leadership Development (SELD) Program as implemented at 
the University of Georgia. The SELD program builds around the skills 
individuals and groups in cooperative extension work need to be effective in 
the future. The centerpiece of SELD is the Managerial Assessment of 
Proficiency (MAP) , the assessment portion of which is a video-driven, 
competency-based, computer-scored simulation of 200 items that assesses the 
participant's proficiency in 12 competencies. Two basic calculations are 
required to compute ROI: the benefit/cost ratio and the second ROI. 
Calculating ROI is not easily done, but with the access to personnel records 
possible in this evaluation, the analysis was made somewhat easier. Data were 
available for 185 Cooperative Extension Agents. Findings suggest that for 
every $1 invested, $3.86 in benefits were returned, and for each $1 invested 
in the program, there is a return of $2.86 in net benefits. Good record 
keeping makes the calculation of ROI much easier. (SLD) 
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INTRODUCTION 



Measuring the Return On Investment (ROI) in training and development has consistently earned 
a place among the critical issues in the Human Resources Development (HRD) field. The topic 
appears routinely on conference agendas and at professional meetings. Journals and newsletters 
regularly embrace the concept with increasing print space. At least a dozen books provide 
significant coverage of the topic. Even top executives have developed an appetite for ROI 
information (Phillips, 1 997). Leadership educators may soon find that program sponsors and 
administrators will be asking for Return On Investment information as well. 

Although the interest in the topic has heightened and much progress has been made, it is still an 
issue that challenges even the most sophisticated and progressive HRD departments and those 
involved with leadership development programs. Some professionals argue that it is not possible 
to calculate the ROI of many programs, while others develop measures and ROI calculations. 
Regardless of the position taken on the issue, the reasons for measuring the return are still 
there(Phillips, 1977). Most professionals involved in training and development share a concern 
that they must eventually show a return on their training investment and thereby abandon some of 
the more traditional methods of evaluating programs. 

The term evaluation has been used by leadership educators and human resource professionals in a 
variety of ways. Most professionals would agree that the term evaluation implies a ‘change in 
something’ or connotes the value or worth of a program or training. How one measures the 
‘change’, ‘value or worth’ varies greatly, however, most evaluations seem to fall into one of four 
categories. A survey conducted by Training in 1996 surveyed over 40,000 training managers and 
specialists to determine the status of how training was evaluated. Table 1 shows the results. 



Table 1: Evaluation of Training in Industry 



Level 


As a Percent of 
Organizations Measuring 
at this Level 


Percent of 
Courses Measured 
at this Level 


Level 1 : Training Reaction 


86% 


83% 


Level 2; Learning 


71% 


51% 


Level 3; Behavior 


65% 


50% 


Level 4: Business Results 


49% 


44% 



Source: Training magazine, October 1996, p. 63. 
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As indicated in table 1, most evaluations were conducted at the lower levels -measuring a 
participants reaction to a program (level 1), or measuring skills, knowledge or attitude changes 
(level 2). Measuring behavioral changes (level 3) and the business impact of the program (level 
4) were the two remaining levels of evaluation. The report indicates that surprisingly, these last 
two levels ( 3 and 4) accounted for 65% and 49% respectively of the evaluations organizations 
conducted. Phillips(1997) states that ''there is a distinct trend toward more accountability of 
training, particularily at the higher levels of evaluation where training is connected to business 
results . As a result of this trend Phillips(1997) states that "a fifth level of evaluation has 
become more prominent-Retum on Investment or ROI.” This level of evaluation compares the 
monetary value of the results with the cost for the program and is usually expressed as a 
percentage figure. 

Evaluation of leadership programs have largely been limited to the lower levels of evaluation. 
The W.K. Kellogg Foundation (2002) reviewed 55 major leadership development programs with 
the hope that it would shed some light on how various programs are evaluating outcomes and 
impacts. Their key findings indicate that most programs measured individual leadership 
outcomes in the traditional ways-participant reactions, changes in knowledge, skills, attitudes, 
behaviors, relationships built, etc. Few dealt with the monetary value of the program— comparing 
results with cost or ROI. At best, some programs reported leveraged dollars or money that 
individuals, collaborations of program participants, organizations or communities have been able 
to attracted as a result of the program. The implication with such a measure is that these dollars, 
being attracted as a result of the program implies an improved leadership capacity. 

Regardless of the setting, ROI is now taking on increased interest (Phillips, 1997). Executives 
and program directors who watched their training budgets grow without appropriate 
accountability measures are now demanding a return on training investment. Leadership 
educators are not exempt from this dilemma. Most are being asked to report impacts, more will 
be asked to report return on investment. 



This paper reports the return on investment of the Southern Extension Leadership Development 
(SELD) program as implemented at The University of Georgia. A brief explanation of that 
program is provided below followed by the calculations used to determine the ROI on the 
Georgia SELD program. 

SELD : Southern Extension Leadership Development 

During the past decade, the Cooperative Extension System has faced an era of economic scarcity 
and has been impacted by a number of internal and external challenges (Ladewig & Rohs, 2000). 
Many of these changes and challenges have changed the nature of work and relationships. 
Organizations that respond to the changing nature of work and authority relationships are 
learning organizations (Senge, 1990). 

A major challenge impacting the transition to a learning organization is that few Extension 
administrators are professionally trained in competencies and styles of leadership appropriate for 



learning organizations. Rather, they have been promoted to leadership positions because they 
excelled in their subject-matter discipline, and they learn their new craft by emulating those who 
proceeded them. While this practice is commonplace throughout the industrialized world, these 
administrators often lack the necessary leadership competencies necessary to truly transform 
their organizations to compete in the information technology era (Patterson, 1998). 

In response to the growing need to understand and cope with the many changes currently and 
potentially impacting the Extension System, Cooperative Extension Directors and Administrators 
of the Southern Region called for the establishment of a regional leadership development 
program. The results was the formation of Southern Extension Leadership Development (SEED). 

The SEED program is unique in that the competency-based approach builds around the skills 
individuals and groups in Cooperative Extension need to be effective in the future. With such 
knowledge. Extension educators can design professional development plans that are relevant, 
useful, and customized to their needs. While regional workshops were conducted individual 
states were encouraged to implement their own leadership development program. The 
centerpiece of SEED is the Managerial Assessment of Proficiency (MAP), developed by 
Training House, Inc. of Princeton, NJ. The assessment portion is a video-driven, competency- 
based, computer-scored simulation consisting of 200 items that assesses a participarit's 
proficiency in 12 competencies. The twelve competencies are: Time management, setting goals, 
planning and scheduling work, training, coaching and delegating, appraising people and 
performance, disciplining and counseling, listening and organizing, giving clear information, 
getting unbiased information, solving problems, making decisions and weighing risk, thinking 
clearly and analytically. The assessment portion was followed by a series of competency 
building workshops to strengthen participants weaker competency areas. 

PROGRAM COSTS AND BENEFITS 

Before calculating the Return On Investment program costs and benefits must be determined. 
Tabulating the costs involves monitoring or developing all related costs of the program targeted 
for the ROI calculation. Among the cost components that should be included are: 

* cost of the design of the program, prorated if possible over the expected life of the 
program; 

* cost of all program materials provided to participants; 

* instructor costs, including preparation time and delivery time; 

* costs of facilities for the program; 

* travel, lodging and meal costs for participants, if applicable; 

* salaries, plus employee benefits of the participants who attend the program; 

* administrative and overhead costs of the training function. 

Phillips(1997) states that specific costs related to the needs assessment and evaluation should be 
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included, if appropriate. Phillips(1997) recommends the conservative approach, including all of 
these costs so that the total is fully loaded. 

Program benefits, especially for leadership educators , may be harder to measure. Converting 
leadership impact data into monetary values, in most instances is difficult-hut not impossible. 

For example, a team building program in a manufacturing plant reduced the number of monthly 
grievances filed by employees. Six months after the program was completed the reduction in 
grievances per month (related to the program as determined by supervisors) declined by seven. 
Using the labor relations staff in the company, the cost of the average grievance was estimated to 
be $6,500 when considering time and direct costs. At seven grievances per month the total annual 
value of the program would be $546,000 ( 84 x $6,500= $ 546,000). 

When converting data to monetary benefits one must first focus on a unit of improvement or 
measurement. In the above case that unit was the number of grievances per month. Then a value 
was assigned to each unit or grievance. In the above case the value or cost of one grievance was 
estimated by those most familiar with the process, i.e. the labor relations staff. Once these figures 
are determined, the value of the program or improvement can be calculated. 



Calculating the Return on Investment (ROI~) 

Two basic calculations are required to compute the Return On Investment. The first is the 
Benefit/Cost Ratio (BCR) and the second Return on Investment (ROI). Those formulas and an 
example follow. 

BCR = Program Benefits 
Program Cost 

ROI (%) = Net program Benefits ^ j qq 
P rogram Costs 

The BCR utilizes the total benefits and costs. In the ROI formula, the costs are subtracted from 
the total benefits to produce net benefits which are divided by the costs. For example, a program 
at Peach State, Inc. produced benefits of $283,500 with a cost of $40,500. Therefore, the 
benefit/cost ratio is: 



BCR = $283,500 = 7.0 (or 7.0:1) 

$40,500 

As this calculation shows, for every $1 invested, $7 in benefits are returned. In this example, net 
benefits are $283,500 - $40,500 = $243,000. Thus, the ROI is : 

ROI(%) = p43,000 X 100 = 600% 

$ 40,500 



This means that for each $1 invested in the program, there is a return of $6 in net benefits. The 
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benefits are usually expressed as annual benefits, representing the amount saved or gained for a 
complete year after program completion. While the benefits may continue after the first year if 
the program has long-term effects, the impact usually diminishes and is omitted from 
calculations. 



ROI: A LEADERSHIP EXAMPLE 

New Extension Agents hired between 1995 and 2001 who completed their probationary period of 
18 months with the cooperative extension service formed the population for this study. Based on 
Extension Personnel records a total of 216 new county extension agents (CEA’s) were hired and 
participated in the same new employee orientation program. Only 185 CEA’s completed their 
probationary period during this time and were included in this study. Basic background data was 
also gathered on these employees. This data included gender, degree level (BS, MS), ethnic 
background and job responsibility (AG, FACS, 4-H) and if resigned or terminated. Based on 
personnel records these new CEA’s were then grouped into one of two groups. The first group 
were those individuals who completed the Georgia SELD/MAP and follow up Excel skill 
building program workshops and the second group— those who did not participate in the 
SELD/MAP and follow up program. It was found that 40 CEA’s completed the Georgia 
SELD/MAP program and 145 did not. Furthermore only two CEA’s in the SELD/MAP group 
had left the organization and 38 of the CEA’s in the non-SELD/MAP group had left the 
organization during this time period. These figures represent an employee turnover rate of 5% 
and 26% respectively. It was this difference in turnover rates ( 26% vs. 5%) that provided the 
basis for calculating the program benefits in monetary terms. 

Program Benefits 

Based on these figures the potential turnover rate among the SELD/MAP participants could have 
been as high as 26% or 10 CEA’s leaving instead of the actual 2 CEA’s at the 5% turnover rate. 
Thus 8 (10-2=8) fewer CEA’s left the organization benefiting the Georgia Extension 
organization in the amount of $400,000 ($50,000 x 8 = $400,000). 

Program Co.sts 

To calculate the ROI on the Georgia SELD/MAP program the following program cost 
components were used: 

* Program Design Costs- license fee, costs for video tapes, etc. per individual 

@$10.00. Total costs= $400.00 



* Program Material Costs to Participants- @ $60.00 per participant for MAP and $100.00 
per participant for follow up Excel workshop. Participants=40. 

Total Costs= $2400.00 + $4,000.00 = $6,400.00 
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* Instructor Costs- prorated salary time @ $500.00 per day (includes all fringe benefits and 

costs to the organization/university). Two instructors -12 days per year for 
6 years. Total Costs= $36,000. 

* Facilities Costs- State 4-H Center charge for use of facilities/equipment @ $8.00 per 

participant for follow up Excel workshop. ($8.00 x 40= $320.00) 

Total Costs = $320.00 

* Travel Costs- 28 cents / mile for auto, $28.00 per day for meals and $55.00 per night for 

lodging if necessary ( includes instructor and participant travel costs). 
Amounts based on attendance and travel records. 

Total Costs= $10,460.00 

* Salaries plus employee benefits for participants- based @ $50,000 per year- prorated 

For number of days in the program ( 6 days). The $50,000 figure was 
arrived at based on administrative communications and college business 
office calculations. Cost computation- $200.00/ day x 6 days= $1200/ 
participant. $1200.00 x 40 participants= $48,000.00 

* Administrative and Overhead Costs- @ 2% of total cost = $2,031.00 

(Accounting figure used by college business office) 

TOTAL PROGRAM COSTS = $103,611-00 



Using the formulas for BCR (Benefits/cost Ratio) and ROI (Return on Investment) the following 
calculations were made: 



TOTAL BENEFlTS-$400,000 
TOTAL PROGRAM COSTS -$103,611 

BCR- $400,000/ $103,611 =3.86 

ROI- ($400,000-$103,611) / $103,611 x 100 = 286% 

Thus the benefit cost ratio (BCR) calculation shows that for every $1 invested , $3.86 in benefits 
are returned and that for each $1 invested in the program, there is a return of $2.86 in net 
benefits. 



DISCUSSION AND SUMMARY 

Calculating the Return On Investment of a leadership program is not easily done. Many programs 
are not able to convert outcome measures to monetary terms. In this particular case one measure 
could be used-employee turnover. Identifying a unit of measurement that can be converted to a 
monetary value will present a challenge to those involved in leadership training. While business 
and industry can provide some examples that may help with identifying outcome measures easily 



converted to monetary value many leadership programs are conducted in settings less conducive 
to assigning monetary values to outcomes with any degree of validity. Fortunately in this study 

personnel records and travel data were either easily accessible or readily computed with relative 
ease. 

The access to personnel records, while not complete, allowed for some additional data analysis. 
Specifically, were the two groups of extension agents ( MAP/SELD participants vs non- 
participants) significantly different from each other on some variable that might influence the 
results? To answer this question some basic background data was available from their files. The 
data that was analyzed included major job responsibility ( agriculture, family/consumer science 
or 4-H youth), gender, ethnicity, degree level ( BS, MS) and whether or not they were still 
employed by the Georgia Extension Service. The only statistically significant difference (p< .01) 
found between the two groups was that those who did not participate in the MAP/SELD program 
were more likely to leave the organization than those who did (Table 2.). 

Table 2: Test of significance between groups on selected variables 

Variable Mean 

Variable Map/Seld No Map/Seld t-value 

N=40 N=145 



Job Responsibility 
(2 = Agr, 3 = FACS, 
4 = 4-H Youth) 


3.10 


3.20 


-0.37 


Gender 

(1 = male, 2 = female) 


1.60 


1.60 


1.00 


Ethnicity (1 = Caucasian, 2 
= African American, 3 = 
other) 


1.12 


1.05 


1.34 


Degree Level 
(1 =BS, 2 = MS) 


1.49 


1.50 


0.96 


Resign/Terminate 
(1 = yes, 2 = no) 


1.93 


1.73 


-2.65* 


♦p<.01 



This study indicates that for every $1.00 spent in the MAP/EXCEL program $3.86 in benefits are 
realized and $2.86 in net benefits are returned on the investment. Had all those extension agents 
hired between 1995 and 2001 participated in the program, employee turnover might have been 
reduced by 21% , saving the organization a minimum of $1,550,000.00 (31 fewer terminations x 
$50,000— $1,550,000) in annual employee turnover costs. Since these terminations occurred after 
an average of 2.33 years of employment the total savings to the Georgia Cooperative Extension 
Service could have exceeded $3.62 million ($1,550,000 x 2.33 years). 
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While this study does not account for all the variables that may influence whether a person 
continues with the organization or not, one of the key factors can be the type of management and 
leadership training they receive. In some instances a monetary value can be calculated to show 
the benefits of such a program. With good record keeping program costs can also be determined. 
With benefit and costs figures the return on investment (ROI) on a program can be easily 
computed. 
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